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ABSTRACT 

whether graphic organizers (GOs) were more effective than concept 
maps (CMs) in facilitating text learning. Experiment 1, which 
involved 56 undergraduates, was a replication of the second 
experiment of D. H. Robinson and K. A. Kiewra (1995) except that only 
GO and CM conditions were included. All materials were constructed by 
outside sources to help avoid researcher bias. Experiment 2, with 46 
undergraduates, used a different text and separate GO, CM, and test 
constructors. Experiment 3 replicated Experiment 2 except that the GO 
constructor constructed the tests. Results show that GOs only 
facilitated text learning when they were constructed by the person 
who also constructed the tests. The implications of this study are 
that adjunct displays will probably not be useful for classroom 
learning unless the tests match the types of learning the displays 
are intended to facilitate. (Contains 15 references.) (Author/SLD) 
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Three experiments were conducted to investigate whether graphic organizers (GO’s) 
were more effective than concept maps (CM’s) in facilitating text learning. Experiment 1 was a 
replication of Robinson and Kiewra’s (1995) second experiment except that only GO and CM 
conditions sere included. Experiment 2 used a different text, and separate GO, CM, and tests 
constructors to reduce experimenter bias. Experiment 3 replicated Experiment 2 except that the 
GO constructor constructed the tests. Results showed that GO’s only “facilitate” text learning 
when they are constructed by someone who also constructs the tests. The implications of this 
study are that adjunct displays will probably not be useful for classroom learning unless the tests 
match the types of learning the displays are intended to facilitate. 
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Introduction 

The traditional method of helping students to learn concept relations has been to supplement text 
material with outlines. Graphic organizers, traditionally known as advance organizers (Ausubel, 1960) use 
a linear or if preferred, a two-dimensional presentation to convey concept relations (Bransford & Johnson, 
1972). The use of graphic organizers to facilitate reading comprehension has been the subject of numerous 
research investigations. Knowledge of text structure, training in the use of text structure, and the use of 

graphic organizers in content area reading has been provided by current research practices in education 
and psychology. 

There are different types of Graphic organizers. A matrix (Kiewra, Dubois, Christian, & McShane, 
1988) uses rows and columns to represent a concept and corresponding attribute values. Another type of 

visual-spatial organizer is a concept map, which uses arrows and lines to connect related concepts without 
the reference attributable values. 

Kruley, Sciama, and Glenberg (1994) found that when students attempted to comprehend text that 
were accompanied with pictures as compared to texts that have no pictures, their performance on a spatial 
short-term memory task was diminished. In a similar study Katayama, Robinson and Fan (1995) found 
that concurrent tasks involving the visuo-spatial sketchpad and the phonological loop support the notion 
that graphic organizers are viewed like a picture. 

Many implications are made through the use of graphic organizers, especially in test taking 
situations. Because many objective tests are circulated through the knowledge of operant concepts, the use 
of a visual-spatial system for taking lecture notes on these concepts has also been proven to increase test 
scores (Kiewra & DuBois, 1992). 

This study consisted of two experiments similar in the independent variables (e g., type of study 
material) but different among the dependent variables (e.g., the actual test over the material). With time 
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constraints and the use of a questionnaire in the second study, we were able to assess how the students felt 
about the “reader friendliness” of the test and materials as well as respond to the time factor. 

Participants and Design of Experiment 1 

Methodology 

Fifty-six undergraduate educational psychology students participated in this experiment in 
exchange for extra-credit in their undergraduate courses. Each student signed a consent form and was 
randomly assigned to one of three conditions (text only, text with concept map, or text with graphic 
organizers). 

Materials 

All materials were constructed by outside sources to help avoid any researcher bias in this 
experiment. All outside sources agreed to contribute the material for the purposes of both experiments. All 
materials remained consistent with the text material that was presented in class. 

Text. A section of text taken from Applying Educational Psychology (Dembo, 1 994) was used. It 
consisted of approximately 5,500 words (seven pages, single spaced) over the topic of "classroom 
instruction and evaluation". One version was used for all three groups. 

Concept Map . A one page map consisting of about 600 words, 30 arrows (to show relationships 
and 40 concept nodes (with main ideas, concepts, etc.) was used as supplemental study material for one of 
the assigned groups. This concept map (Jonassen, 1994) was used with the acknowledgment of 
Pennsylvania State University. 

Graphic organizers . A seven page study supplement consisting of rows and columns was used 
with the corresponding text. It covered the same material as the text and the concept map. To prevent 
researcher bias of study aid, the graphic organizers (Kiewra and Dubois, 1995) were used with permission 
already granted by the creators at the University of Nebraska and the State University of New York 
respectively 
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Tests . To assure unbiased instrumentation, the tests have also been constructed by an outside 
source. The tests were constructed by Armbruster (1995) and measured facts, relations, and application 
covered in Dembo's segment of text. The tests were derived upon the text alone as no spatial displays were 
involved. The tests were used with permission already granted by the originator. The test itself consisted 
of twenty-four questions: six short answer, two multiple choice, two short essay, and sixteen fill-in-the- 
blank items. 

Procedure 

Studying took place during two separate class sessions. In the first session, subjects were handed a 
9" x 12" coded envelope containing one of the three study materials and a blank piece of paper to make 
any notes they wish. The subjects were asked to quietly read their study materials completely. This session 
lasted approximately 45 mins, after which the students turned in their materials inside of their coded 
envelopes upon being dismissed. 

Session two took place two days after the first. The students were given their same study material 
and were allowed ten mins, to review their material before taking the test. When approximately 45 mins, 
had expired, they were asked to place all their materials inside the 

envelope, write demographic information on the outer tab (e.g., sex, major, GPA), peel off their names 
from the envelope to ensure anonymity. 

Results 

The answer key had also been provided by the originator of the test and was used according to the 
provided guidelines. Four tests were eliminated from the evaluation because of incomplete answer sheets. 
For evaluation, each treatment condition was grouped separately and graded accordingly. A 3 study 
material (text vs. concept map vs. graphic organizer) between groups analysis of variance (ANOVA) was 
conducted on the collected data. The statistical package used to analyze the data was conducted on a 
Windows 6. 1 SPSS program. There was no significant main effect of study supplement for type of spatial 
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display. The analysis yielded mean scores for text only group, (M “ 15.50, SD = 7. 17, p > .05) which was 
not statistically different from the graphic organizer group (M = 15.06, SD = 15.06, p > .05) and only 
marginally higher than the concept map group (M = 13.75, SD = 5.88, p > .05). There was no significant 
differences between the groups studying the text only and the concept map or for the graphic organizer 
group and the concept map group. 

Conclusion 

This experiment did not support the notion of one type of spatial organizer to be superior to 
another in enabling students to recall information more efficiently. The hypothesis, that students with 
graphic organizers as study supplements would score significantly higher than those with the study 
supplement of concept maps and with no visual-spatial organizers at all (text alone) was actually 
contradicted. A possible explanation for this finding might infer that the test was constructed with some 
bias toward the textual perception and test relativity. Therefore, a logical rationale would suggest that 
because the test constructor had access to only the text and not any of the two spatial displays, the test was 
based exclusively on the text. From what has been researched on the benefits of graphic organizers as 
adjunct displays to text, suggestions for future studies would want to also investigate the effectiveness of 
graphic organizers on test construction . 

Participants and Design of Experiment 2 

Methodology 

Experiment 2 was very similar to experiment 1 with the exception of the test materials and the 
length of testing. The test materials for this experiment were constructed by a different outside expert. 
Forty-six undergraduate educational psychology students enrolled at Mississippi State University 
participated in this research project in exchange for extra-credit. The students were randomly assigned to 
one of three conditions of a 3 x 2 design. Testing took place in a typical university classroom on the 
campus of Mississippi State University. 
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Materials 

All study materials were the same as experiment 1. All precautions were made to assure subject 
confidentiality. Again, all precautions were made to prevent researcher bias on any of the testing material. 

Text. The same as experiment 1 . 

Concept map . The same as experiment 1 . 

Graphic organizers. The same as experiment 1 . 

Tests. To assure unbiased instrumentation, the tests were constructed by another outside source. 
This time the tests were constructed by Kiewra (1995). These tests measured factual and conceptual 
relationships covered in Dembo's segment of text material. The tests were used with permission granted 
by the originator. The test consisted of forty-three multiple choice and short answer questions: thirty-three 
factual and ten conceptual. Each question carried a value of one point. There were a total of forty-three 
possible points on the test. This test contained more items than the test used in experiment 1 but took 
about 10 less min. to complete. 

Questionnaire . A one page questionnaire consisting of 7 multiple choice questions were given to 
each student. These questions were developed to assess the students prior knowledge of the content, and 
various questions seeking the students opinion of the study materials such as reader friendliness of the test 
and materials, level of interest of the material, etc. 

Procedure 

The same procedures were used as in experiment 1 for both days. The only variation was about ten 
min. in time. 

Dav 1. The first session consisted of a 35 min. reading period in which students were asked to 
quietly read over the material within their respective envelopes. After approximately 35 min., the students 
were asked to place their materials inside of their coded envelope and turn them into the researcher. 
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Day 2. The second session took place two days after the first. The students were given the exact 
same envelope containing their study materials and were allowed ten mins, to review their material. Then 
the students were given a test that lasted about 25 min. to complete and were instructed to fill out the 
provided questionnaire consisting of 7 multiple choice items which took approximately five min. to 
complete. Upon completion, the students were asked to place their materials into their envelopes, peel 
their coded labels and turn in their envelopes. The answer key had also been provided by the originator of 
the test and was used according to the provided guidelines. 

Results and Discussion 

Each treatment condition was grouped separately and graded accordingly. A 3 study condition 
(text, concept map or graphic organizer) between groups mixed model analysis of variance (ANOVA) was 
used to analyze the data upon both factual and conceptual test items as the within groups condition. The 
results yielded a significant difference for the graphic organizer group (M = 19.53, SD = 6.5 1, p < .05) 
than both the text only group (M = 14.75, SD = 3. 15, p > .05) and the concept map group (M = 16.40, SD 
= 2.56) on the factual items. For the conceptual items, the graphic organizer group (M = 5.93, SD = 1.94, p 
< .05) scored significantly higher than the text only group (M = 4.00, SD = 1.83) but only marginally 
higher than the concept map group (M = 4.80, SD = 2.0 1 , p > .05). There was no difference between the 
text only and the concept map groups. The total scores between the three groups yielded a significant 
higher scores for the graphic organizer group (M = 25.47, SD = 7.85, p < .05) than the text only group (M 
= 18.75, SD = 4. 19, p < .05) and the concept map group (M = 21.20, SD = 2.93, p< .05). The concept map 
group scores were marginally higher than the text only scores. 

In addition, the researchers added several of the variables derived from the questionnaire into the 
analysis to see if there may be an effect of those variables upon the test scores. There was no impact of 
level of effort reported by the students and conceptual items and on the total score of the tests. Although 
not significant, within the factual questions, the least amount of effort response scored marginally higher 
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than the reasonable and considerable amount of effort responses. For the interest levels reported by 
students, there were no differences among the conceptual and total item scores whereas the factual items 
observed a higher score for responses regarding more interest. For the students report of prior knowledge, 
the concept and total items observed higher scores toward the greater the greater the students prior 
knowledge. However, for the factual items, students scored higher when they viewed the information as 
being totally novel. There was no impact of reader friendliness of the material and test scores. 

Conclusion 

As we suspected, the type of test that is constructed and by whom the test was constructed would 
impact the results of the three groups. In experiment 1 we observed that the text only group had a slight 
advantage in that the test was constructed by an outsider who only had the chapter length text to go by. 
Even though the results were not significant, they indicated a slight edge toward the text only condition. 
The same notion holds true in experiment 2 where this time the test was constructed by the same outsider 
who constructed the graphic organizers. With this method, we predicted that the students in the graphic 
organizer group would have an advantage and outperform the other groups on the test. And as a result we 
found this was true. Perhaps the results would suggest that the type of spatial display is of secondary 
nature to what is perceived to be important by the test constructor based upon their displays. Further 
research might investigate the same variables and use the constructor of a concept map to generate a test 
over the overall material. If this suggested study should yield similar results in favor for the concept map 
group then the hypothesis of test constructor bias on tests would be strengthened. Also, to what degree the 
differences between groups are might indicate which spatial display is more effective in conveying the 
information being presented for testing purposes. 



RESEARCHER-CONSTRUCTED DISPLAYS 9 



References 

Armbruster, B.A. Testing material (personal unpublished contribution, December, 1 994). 

Ausubel, D.P. (1960). Educational psychology; A cognitive 
view. New York: Holt & Co. 

Bransford, J.R., & Schooler, L.J. (1991). Reflections of the environment in memory. 

Psychological Science. 2. 396-408. 

Dembo, M.H. (1994). Applying Educational Psychology, fifth edition (excerpt from text, pp. 551- 
557). New York: Longman. 

Jonassen, D. Concept Map (personal unpublished contribution, February, 1995). 

Katayama, A.D. & Robinson, D.H. (1996, April). Which adjunct displays help students learn best? 
A comparison of the effectiveness of researcher-constructed displays . Paper accepted for presentation at 
the annual meeting of the American Educational Research Association, New York. 

Katayama, A.D. & Fan. A C. (1995, October). Evidence for use of the visuo-spatial sketchpad in 
processing graphic organizers. Paper presented at the annual meeting of the Mid-South Educational 
Research Association, Biloxi, MS. 

Keller, R.J. (1983). Using graphic organizers to improve lecture notetaking behavior in 
community college students. Unpublished doctoral dissertation, Northern Illinois University. 

Kiewra, K.A., & Dubois, N.F. Graphic organizers and Test (personal unpublished contribution, 
March, 1995). 

Kiewra, K.A., Dubois, N.F., Christian, D., McShane, A., et al. (1991). Note-taking functions and 
techniques. Journal of Educational Psychology, 83. 240-245. 

Kruley, P., Sciama, S.C., & Glenberg, A M. (1994). On-line processing of textual illustrations in 
the visuospatial sketchpad: Evidence from dual-task studies. Journal of Memory and Cognition. 22. 261- 
272. 




li 



RESEARCHER-CONSTRUCTED DISPLAYS JQ 



Mayer, R.E. (1984). Aids to text comprehension. Educational Psychologist, 19. 30-42. 

Robinson, D.H., Katayama, A.D., & Fan, A.C. (1995, submitted). Evidence for use of the visuo- 
spatial sketchpad in processing study materials as adjuncts to text. Contemporary Educational Psychology 
Robinson, D.H. & Schraw, G. (1994). Computational efficiency through visual argument: Do 
graphic organizers communicate relations in text too effectively? Contemporary Educational Psychology. 
19,399-415. 

Tulving. E. (1983). Elements of episodic memory . London: Oxford University Press. 



4 



7 Ty O ^ 



AERA April 8-12, 1996 




U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement (OERI) 
Educational Resources Information Center (ERIC) 

REPRODUCTION RELEASE 

(Specific Document) 




I. DOCUMENT IDENTIFICATION: 



T,tle Which Adjunct Displays Help Students Learn Best? 
Effectiveness of Researcher-Constructed Displays 


A Comparison of the 


Authors): Andrew D. Katayama, Daniel H. Robinson, Bonnie Armbruster, Nelson F. Dubois, 
Judith G. Groulx, David R. Jonassen, Kenneth A. Kiewra, and William D. Winn 


Corporate Source: 

Department of Counselor Education/ Educational Psychology 
Mississippi State University 


Publication Date: Paper presente 
at AERA, New York 



II. REPRODUCTION RELEASE: 



In order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents 
announced in the monthly abstract journal of the ERIC system, Resources in Education (RIE), are usually made available to users 
in microfiche, reproduced paper copy, and electronic/optical media, and sold through the ERIC Document Reproduction Service 
(EDRS) or other ERIC vendors. Credit is given to the source of each document, and, if reproduction release is granted, one of 
the following notices is affixed to the document. 

If permission is granted to reproduce the identified document, please CHECK ONE of the following options and sign the release 
below. 



y 



Sample sticker to be affixed to document Sample sticker to be affixed to document 



Check here 

Permitting 
microfiche 
(4"x 6" film), 
paper copy, 
electronic, 
and optical media 
reproduction 



"PERMISSION TO REPRODUCE THIS 




"PERMISSION TO REPRODUCE THIS 


MATERIAL HAS BEEN GRANTED BY 




MATERIAL IN OTHER THAN PAPER 






COPY HAS BEEN GRANTEO BY 








TO THE EDUCATIONAL RESOURCES 






INFORMATION CENTER (ERIC)." 




TO THE EDUCATIONAL RESOURCES 






INFORMATION CENTER (ERIC)." 


Level 1 


Level 2 



or here 

Permitting 
reproduction 
in other than 
paper copy. 



o 

ERLC 



Sign Here, Please 

Documents will be processed as indicated provided reproduction quality permits. If permission to reproduce is granted, but 
neither box is checked, documents will be processed at Level 1. 



“1 hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce this document as 
indicated above. Reproduction from the ERIC microfiche or electronic/optical media by persons other than ERIC employees and its 
system contractors requires permission from the copyright holder. Exception is made for non-profit reproduction by libraries and other 
service agencies to satisfy information needs of educators in response to discrete inquiries. 


Signature: /j ^ — - 


Position: 

Graduate Research Assistant 


Printed Name: /J /J 


Organization: 


Andrew D. Katayama 


PREPS, Inc. 


Address: 


Telephone Number: 


P.0. Box 5365 


( 601 ) 325-7060 


Mississippi State, MS 39762 


Date: 

April 3, 1996 



